Papers with Qualitative analysis
Rhetorical Questions in LLM Representations: A Linear Probing Study (2026.acl-long)
Copied to clipboard
| Challenge: | Rhetorical questions are asked not to seek information, but to persuade or signal stance . how large language models internally represent rhetorical questions remains unclear . |
| Approach: | They analyze rhetorical questions in LLM representations using linear probes on two social-media datasets with different discourse contexts. |
| Outcome: | The results show that rhetorical signals emerge early and are most stably captured by last-token representations. |
Communication Enables Cooperation in LLM Agents: A Comparison with Curriculum-Based Approaches (2026.eacl-short)
Copied to clipboard
| Challenge: | a "cheap talk" channel increases cooperation in 4-player Stag Hunt, but a complex curriculum can induce "learned pessimism" in agents. |
| Approach: | They investigate whether a direct communication channel can elicit cooperation in multi-agent LLMs. |
| Outcome: | The proposed curriculum reduces agent payoffs by 27.4% in a 4-player Stag Hunt simulation. |
GraphRAG-Rad: Concept-Aware Radiology Report Generation via Latent Visual-Semantic Retrieval (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing encoder-decoder models suffer from hallucinations, generating plausible but incorrect medical findings. |
| Approach: | They propose a novel architecture that integrates biomedical knowledge through a latent visual-semantic retrieval approach. |
| Outcome: | The proposed architecture achieves competitive performance with strong results across multiple metrics. |
Cross-Lingual Abstract Meaning Representation Parsing (N18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) research has focused on English . Qualitative analysis shows that the new parsers overcome structural differences between the languages. |
| Approach: | They propose to use an AMR parser for English and parallel corpora to learn AMR for Italian, Spanish, German and Chinese. |
| Outcome: | The proposed method overcomes structural differences between the target languages and requires no gold standard data. |
Evolutionary Search for Automated Design of Uncertainty Quantification Methods (2026.acl-srw)
Copied to clipboard
Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii, Oleg Somov, Alexander Panchenko, Elena Tutubalina
| Challenge: | Qualitative analysis reveals that different LLMs employ qualitatively distinct evolutionary strategies for automating, interpretable hallucination detector design. |
| Approach: | They apply LLM-powered evolutionary search to discover unsupervised UQ methods represented as Python programs and apply them to atomic claim verification. |
| Outcome: | The proposed methods outperform strong manually-designed baselines while generalizing robustly out-of-distribution. |
ComfyUI-R1: Exploring Reasoning Models for Workflow Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | ComfyUI-R1 is the first large reasoning model for automated workflow generation. |
| Approach: | They propose a large reasoning model for automated workflow generation that builds on curated knowledge bases and a two-stage framework to fine-tune models for cold start and reinforcement learning for incentivizing reasoning capability. |
| Outcome: | The proposed model achieves 97% format validity rate, high pass rate, node-level and graph-level F1 scores, surpassing prior state-of-the-art methods that employ leading closed-source models such as GPT-4o and Claude series. |
Headline Token-based Discriminative Learning for Subheading Generation in News Article (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models that generate news subheadings rely on topical headline information to capture topical knowledge from the article. |
| Approach: | They propose a model that uses topical headline information to generate news subheadings using masked headline tokens. |
| Outcome: | The proposed model outperforms the comparative models on three news datasets written in two languages and performs robustly on a small dataset and various masking ratios. |
Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA frameworks that use event-centric reasoning are lacking. |
| Approach: | They propose a novel QA model with contrastive learning and invertible event transformation . they use an invertable transformation matrix to project event vectors into a common event embedding space . |
| Outcome: | The proposed model achieves 8.4% gain in token-level F1 score and 3.0% gain in Exact Match score on the ESTER dataset. |
Measuring Mechanistic Independence: Can Bias Be Removed Without Erasing Demographics? (2026.eacl-long)
Copied to clipboard
| Challenge: | Using multi-task evaluation, we examine how independent demographic bias mechanisms are from general demographic recognition in language models. |
| Approach: | They compare attribution-based and correlation-based methods for locating bias features in language models to find out which features are independent from general demographic recognition. |
| Outcome: | The proposed method reduces bias without degrading recognition performance. |
Improving In-Context Few-Shot Learning via Self-Supervised Training (2022.naacl-main)
Copied to clipboard
Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, Zornitsa Kozareva
| Challenge: | Existing approaches to improve in-context few-shot learning are pretraining and downstream fewshot evaluation. |
| Approach: | They propose to use self-supervision as an intermediate training stage between pretraining and downstream fewshot usage to train models to perform in-context few shot learning. |
| Outcome: | The proposed model outperforms baseline models on two benchmarks. |
People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text (2025.acl-long)
Copied to clipboard
| Challenge: | Qualitative analysis of experts’ free-form explanations shows that while they rely heavily on specific lexical clues (‘AI vocabulary’), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity). |
| Approach: | They hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. |
| Outcome: | The annotators who frequently use LLMs for writing tasks outperform commercial and open-source detectors even without evasion tactics like paraphrasing and humanization. |
BERT Rediscovers the Classical NLP Pipeline (P19-1)
Copied to clipboard
| Challenge: | Pre-trained text encoders have advanced the state of the art on many NLP tasks . Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically . |
| Approach: | They aim to quantify where linguistic information is captured within a network model . they aim to use pre-trained text encoders to displace static word embeddings . |
| Outcome: | The proposed model can adjust the pipeline dynamically, revealing lower-level decisions on the basis of disambiguation from higher-level representations. |
Multilingual Detection of Personal Employment Status on Twitter (2022.acl-long)
Copied to clipboard
| Challenge: | Detecting disclosures of individuals’ employment status on social media is a challenging task due to their rarity in a sea of social media content and the variety of linguistic forms used to describe them. |
| Approach: | They propose to use BERT-based classification models to identify five types of disclosures about individuals’ employment status in three languages. |
| Outcome: | The proposed methods achieve significant gains in precision, recall, and diversity of results in real-world settings of extreme class imbalance. |
Enhancing Image-to-Text Generation in Radiology Reports through Cross-modal Multi-Task Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Image-to-text generation relies on independent models for image understanding and natural language generation, which often exhibit a semantic gap between visual and textual information. |
| Approach: | They propose a multi-task learning framework to leverage both visual and non-imaging data for generating radiology reports. |
| Outcome: | The proposed framework improves performance over single-task baselines across language generation metrics and mitigates overfitting in auxiliary tasks. |
Continuous Decomposition of Granularity for Neural Paraphrase Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Prior work has shown that decomposing sentences at different levels of granularity has improved paragraph generation. |
| Approach: | They propose a model for continuous decomposing granularity for neural paraphrase generation that incorporates granules into attention. |
| Outcome: | The proposed model outperforms baseline models on Quora question pairs and Twitter URLs on two benchmarks. |
A Corpus for Reasoning about Natural Language Grounded in Photographs (P19-1)
Copied to clipboard
| Challenge: | a dataset for visual reasoning with natural language and images is available. |
| Approach: | They propose a dataset for joint reasoning about natural language and images . they crowdsource 107,292 examples of English sentences paired with web photographs . |
| Outcome: | The proposed dataset combines 107,292 examples of English sentences with web photographs . Qualitative analysis shows the data requires compositional joint reasoning . |
The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models (2026.acl-long)
Copied to clipboard
| Challenge: | Mixture of Experts models are widely assumed to achieve domain specialization through sparse routing. |
| Approach: | They propose a framework that analyzes routing behavior at the level of expert groups rather than individual experts. |
| Outcome: | The proposed framework analyzes routing behavior at the level of expert groups rather than individual experts. |
CentaurTA: A Self-Improving Human-Agents Collaboration Framework for Thematic Analysis (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing large language model approaches for qualitative analysis are labor-intensive and costly. |
| Approach: | They propose an iterative human–agent framework for scalable thematic analysis that integrates structured human feedback with rubric-based evaluation. |
| Outcome: | The proposed framework improves coding alignment and transparency across multiple datasets, baselines, and LLM families. |
A Computational Method for Measuring Open Codes in Qualitative Analysis (2026.findings-acl)
Copied to clipboard
John Chen, Alexandros Nikolaos Lotsos, Sihan Cheng, Lexie Zhao, Yanjia Zhang, Jessica Hullman, Bruce Sherin, Uri Wilensky, Michael Horn
| Challenge: | Qualitative analysis is widely adopted across many social science disciplines. |
| Approach: | They propose a theory-informed computational method for measuring inductive coding results from humans and GAI. |
| Outcome: | The proposed method captures breadth, consensus, unique contribution, and systematic deviation without assuming ground truth. |